Network Synchronization in a Noisy Environment with Time Delays: Fundamental 

Limits and Trade-Offs 
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We study the effects of nonzero time delays in stochastic synchronization problems with linear 
couplings in an arbitrary network. Using the known exact threshold value from the theory of differ- 
ential equations with delays, we provide the synchronizability threshold for an arbitrary network. 
Further, by constructing the scaling theory of the underlying fluctuations, we establish the absolute 
limit of synchronization efficiency in a noisy environment with uniform time delays, i.e., the mini- 
mum attainable value of the width of the synchronization landscape. Our results also have strong 
implications for optimization and trade-offs in network synchronization with delays. 
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In network synchronization problems [l|, individual 
units, represented by nodes in the network, attempt to 
adjust their local state variables (e.g., pace, load, ori- 
entation) in a decentralized fashion. They interact or 
communicate only with their local neighbors in the net- 
work, often with the intention to improve global perfor- 
mance. These interactions or couplings can be repre- 
sented by directed or undirected, weighted or unweighted 
links. Applications of the corresponding models range 
from physics, biology, computer science to control the- 
ory, including synchronization problems in distributed 
computing 2], consensus, coordination and control in 
communication networks [3-^], flocking animals [7|, |8J, 
bursting neurons |9l4ll| , and cooperative control of vehi- 
cle formation jl2|. 

There has been a massive amount of research focus- 
ing on the efficiency and optimization of synchronization 
problems [l|, Il3l4l6j in various complex network topolo- 
gies, including weighted [3, [ij] and directed [y, [la, [3 
networks. In this Letter, we study an aspect of stochas- 
tic synchronization problems which is present in all real 
communication, information, and computing networks 
d, 11, [13, mi, including neurobiological networks [lol.[Tlj: 
the impact of time delays on synchronizability and on 
the breakdown of synchronization. The presence of time 
delays, however, will also present possible scenarios for 
trade-offs. Here we show that when synchronization net- 
works are stressed by large delays, reducing local coor- 
dination effort will actually improve global coordination. 
Similarly subtle results have also been found in neuro- 
biological networks with the synchronization efficiency 
exhibi ting non-monotonic behavior as a function of the 
delay [13 [Hi- 

For our study, we consider the simplest stochas- 
tic model with linear local relaxation, where network- 
connected agents locally adjust their state to closely 



match that of their neighbors (e.g., load, or task allo- 
cation) in an attempt to improve global performance. 
However, they react to the information or signal received 
from their neighbors with some time lag (as a result of 
finite processing, queueing, or transmission delays), mo- 
tivating our study of the coupled stochastic equations of 
motion with delay, 
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Here, hi{t) is the generalized local state variable on node 
i and rji (t) is a delta-correlated noise with zero mean and 
variance {r]i{t)r]j{t'))=2DSij6{t — t'), where D is the noise 
intensity. Cij=Cji>0 is the symmetric coupling strength 
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in general weighted networks, where Aij is 



the adjacency matrix and Wij is the link weight). Tij>0 
is the time delay between two connected nodes i and j. 
For initial conditions we use hi{t)^0 for t<0. Eq. ([T|) is 
also referred to as the Edwards- Wilkinson process [22] on 
networks [Sj] with time-delays. Without the noise term, 
the above equation is often referred to as the consensus 
problem [5|, |6| on the respective network. 

The standard observable in stochastic synchronization 
problems, where relaxation competes with noise, is the 
width of the synchronization landscape 0, S \lMi [3 
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where h{t) = i^/N)J2i=ihi{t) is the global average of 
the local state variables and (...) denotes an ensemble 
average over the noise. A network of N nodes is syn- 
chronizable if (w^(oo))<oo, i.e., if the width approaches 
a finite value in the t— >-oo limit. The smaller the width, 
the better the synchronization. 

In the case of uniform delays Tij = r, the focus of this 



Letter, one can rewrite Eq. ([T]) as 
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where Fy — Sij ^j Cu — Cij, is the symmetric network 
Laplacian. In this case, by diagonalizing the network 
Laplacian, one can decompose the problem into N inde- 
pendent modes 



dthk{t) = -Xkhk{t - t) + fjkit) 
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where Afc, fc = 0,l,2,...,iV — 1, are the eigenvalues of the 
network Laplacian and {fik{t)f]i{t'))=2D5ki5{t — t'). For 
a single-component (or connected) network, the Lapla- 
cian has a single zero mode (indexed by fc=0) with Ao=0, 
while Afc>0 for fc>l. Using the above eigenmode decom- 
position, the width of the synchronization landscape can 

be expressed as {w''{t)) = (1/A^) Ef="i'(^fe(0) i- 

For example, for zero time delay (r=0), one immedi- 
ately finds {w^it))M^/N)Y:k=iDX-\l-e~^^^'). The 
above expression explicitly shows that every finite con- 
nected network with zero time delay is synchronizable, 
(w^(oo))<oo. In the limit of infinite network size, how- 
ever, network ensembles with a vanishing (Laplacian) 
spectral gap may become unsynchronizable, depending 
on the details of the small-A behavior of the density of 
eigenvalues [ll-Q. 

In the case of non-zero uniform delays, the case con- 
sidered here, the eigenmodes of the problem are again 
governed by a stochastic equation of motion of identi- 
cal form for all fc>l [Eq. ^]. Thus, understanding the 
time-evolution of a single stochastic variable hk{t) and 
its fluctuations will provide both full insight to the syn- 
chronizability condition of the network-coupled system 
and a framework to compute the width of the synchro- 
nization landscape. Therefore, to ease notational burden 
and to direct our focus to a single stochastic variable, we 
will temporarily drop the index k referring to a specific 
eigenmode, and study the stochastic differential equation 

dth{t) = -Xh{t - t) + 7^(0 (5) 

with {f]{t)fi{t'))=2D&{t — t'). Using standard Laplace 
transformation with initial conditions h{t)—Q for f<0, 
one finds 



h{t)= dt'f,{t')Y.j 
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where Sq, a = 1,2,..., are the solutions of the charac- 
teristic equation 
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in the complex plane. The above complex e quat ion has 
an infinite number of solutions for t>0 [a, l23,l2J|. Using 
Eq. ([6]) , for the noise-averaged fluctuations we find 
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FIG. 1: Time series {h?{t)) for different delays, obtained 
by numerically integrating Eq. ([5]) and averaging over 1,000 
independent realization of the noise. Here, A=l, D=\, and 
Ai=0.01. The theoretical (continuum-time) threshold value 
of the delay (for {h{oo)) to remain bounded) is Tc—t^/2 [25|. 

The solution of Eq. ([7]) with the largest real part gov- 
erns the long-time temporal behavior of the respective 
mode (e.g., stability, approach to, or relaxation in the 
steady state). The condition for {h^{oo)) to remain finite 
is Re(sQ.)<0 for all a. As has been shown for Eq. (O, this 
inequality holds if TA<7r/2 [l,!!!!!]. In Fig. [T] we show 
the time-dependent width of the fluctuations associated 
with a single stochastic variable, obtained by numerically 
integrating Eq. ([5]) for a few characteristic cases |25| . 

Returning to the context of network synchronization, 
synchronizability requires a finite steady-state width, 

(w2(oo)) = (l/iV)Ef^Y(^fe(oo)> < oo- Thus, for uni- 
form time delays in a given network, all k>\ modes must 
have finite steady-state fluctuations (/i|(oo)) < oo. This 
implies that one must have rAfc<7r/2 for all fc>l modes, 
or equivalently |26t . 
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The above exact delay threshold for synchronizabil- 
ity has some immediate and profound consequences 
for unweighted networks. Here, the coupling matrix 
is identical to the adjacency matrix, Cij=Aij, and 
the bounds and the scaling properties of the extreme 
eigenvalues of the network Laplacian are well known. 
In particular, Nk^^y,/{N - l)<Amax<2A:max US], where 
fcmax is the maximum node degree in the network (i.e., 
(Ainax)=C((^max)))- Thus, rA:max<7r/4 is sufficient for 
synchronizibility [26|, while rA:max>7''/2 leads to the 
breakdown of synchronization with certainty. These in- 
equalities imply that networks with potentially large de- 
grees, e.g., scale- free (SF) networks [28 . 29], are rather 
vulnerable to intrinsic network delays [5|, |6| . For exam- 
ple, SF network ensembles with a natural degree cut-off 
exhibit (A„iax)^(fcmax)'^^^/^''~^^ for iV>l (when the av- 
erage degree (fc) is held fixed), where 7 is the exponent 
gov erning the power-law tail of the degree distribution 
[3OJ ■ In turn, the probability that a realization of a ran- 
dom SF network ensemble of N nodes is synchronizable 
approaches zero for any nonzero delay r in the limit of 
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FIG. 2: Steady-state fluctuations {h{oo)) obtained by numer- 
ically integrating Eq. ([5]) as a function of A for different r 
values. Here, D=l, and Ai=0.01. The inset shows the scaled 
plot of the same data points, (h'^{oo))/r vs At, together with 
the numerically fitted scaling function f{x) (solid curve) [25J . 

,-Ari/(7-i)^l. 

Next, we analyze the steady-state behavior of the 
width in the synchronizable regime. We accomplish this 
by investigating the basic scaling features of the steady- 
state fluctuations of a single stochastic variable, (/i^(oo)), 
governed by Eq. ([5]), which can be associated with an ar- 
bitrary mode. In this regime one must have Re(sQ)<0 for 
all a, or equivalently, tX<t:/2. Defining a new variable 
z=Ts^ Eq. ([7]) can be rewritten z + Are~^ = 0, i.e., for 
a given A and r, the solutions for the scaled variable z 
can only depend on Ar, Zq=Zq,(At), a = 1,2,.... Thus, 
the solutions of the characteristic equation Eq. ([7]) must 
exhibit the scaling form Sq = T~^Za(Ar), a = 1,2,.... 
Substituting the above expression into Eq. ([8]) and tak- 
ing the t^>-oo limit immediately yields the scaling form 
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Thus, for a single stochastic variable h{t) governed by 
the stochastic differential equation Eq. ([5]) (simple re- 
laxation in a noisy environment with delay), plotting 
{h?{oo))/T vs Ar (for a fixed noise intensity D) should 
yield full data collapse, as demonstrated in Fig. [5] [25|. 
[While we do not have an analytic expression for the 
scaling function, for small arguments it asymptotically 
has to scale as f{x)^l/x to reproduce the exact limit- 
ing case of zero delay, (/i^(oo))~_D/A. Further, we nu- 
merically found that in the vicinity of 7r/2, it approxi- 
mately diverges as (7r/2 — x)""'^.] The scaling function 
f{x) is clearly non-monotonic; it exhibits a single min- 
imum, at approximately x*w0.73 with f*—f(x*)~3.1. 
The immediate message of the above result is rather in- 
teresting: For a single stochastic variable governed by 
Eq. ([S]) with a nonzero delay, there is an optimal value 
of the relaxation coefficient X*=x* /t, at which point 
the steady-state fluctuations attain their minimum value 
{h? [oo))— Dt J* k.'I^.IDt. This is in stark contrast with 
the zero-delay case where {h?{oo))^D /\, i.e., there the 
steady-state fluctuation is a monotonically decreasing 
function of the relaxation coefficient. 



In addition to gaining fundamental insights, construct- 
ing the scaling function f{x) numerically with some ac- 
ceptable precision of the single variable problem (Fig. [2] 
inset) also provides a method to obtain the steady-state 
width of the network-coupled system: one can numeri- 
cally diagonalize the Laplacian of the underlying network 
and employ the scaling function f{x) to obtain the width. 
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Further, we can now extract the minimum attainable 
width of the synchronization landscape in a noisy en- 
vironment with uniform time delays. For a fixed r, each 
term in Eq. (jlip can be minimized by choosing Afc — x* /r 
for all k>l. Then 
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for large N. This number, the fundamental limit of syn- 
chronization efficiency in a noisy environment with uni- 
form time delays, can be used as a base-line value when 
comparing networks from the viewpoint of synchroniza- 
tion efficiency. Note that there is a trivial network which 
realizes the optimal behavior: the fully connected graph 
with identical coupling constants Cij=x* /Nt for all iy^j- 
(This network has N—l identical non-zero eigenvalues, 
\k=x* /t for all fc>l.) In general, networks with a nar- 
row spectrum centered about A* = x* /t shall perform 
closer to optimal. How to construct such networks with 
possible topological and cost constraints is a different and 
challenging question which we will not pursue in detail 
here, but we note that essentially the same problem arises 
in the broader context of synchronization of generalized 
dynamical systems [l^ [l^ . Recent methods tackling this 
issue involve locally reweighting and/or removing links 
from the networks to achieve optimal performance p^ . 
The essential non-monotonic feature of the scaling 
function f{x) in Eq. (fTTI) (including the potentially di- 
verging contributions from large eigenvalues beyond the 
threshold) presents various trade-off scenarios in network 
synchronization problems with delays. As the simplest 
and obvious application of the above results, consider 
a network which is stressed by large delays beyond its 
threshold, t\^^-^>'it/2 (so that the largest fluctuations 
and the width are growing exponentially without bound). 
Then even a suitably chosen uniform reduction of all cou- 
plings C-j=pCij {X'j,=pXk) with p<Tr/2XjnaxT will lead to 
the stabilization of the system, with a finite steady-state 
width. In communication and computing networks, the 
effective coupling strength Cij can be controlled by the 
frequency (or rate) of local synchronizations through the 
respective link ^. The above results then suggest that 
when the system is beyond its stability threshold, syn- 
chronizing sufficiently less frequently, can lead to stabi- 
lization and better coordination. Figure |3] shows results 
for the case when the communication neighborhood is 
fixed, but the local synchronizations through the links 




FIG. 3: Time dependent width for r=1.27r/2Amax for dif- 
ferent values of the local synchronization rate p on a fixed 
graph, obtained by the numerical integration of Eq. ([T]) with 
At=0.005 and D=1.0. The underlying network is a Barabasi- 
Albert SF graph '2§\ with N=100, (fc>«6, and Amax~32. 

[the coupling terms in Eq. ([T}] are only performed with 
probability p<l, while invoking the noise term at every 
time step. Indeed, reducing the local synchronization 
rate can improve global performance. In fact, even per- 
forming no local synchronizations at all {p=0) leads to 
a slower power-law divergence of the width with time, 
(it;^(i))~2L'i, as opposed to the exponential divergence 
governed by the largest eigenvalue (s) above the thresh- 
old. 

In summary, wc have obtained the delay threshold 



for the simplest stochastic synchronization problem with 
linear couplings in an arbitrary network. Further, by 
exploring and investigating the scaling properties of 
the fluctuations associated with the eigenmodes of the 
network Laplacian, we found the minimum attainable 
steady-state width of the synchronization landscape in 
any network. The non-monotonic feature of the scal- 
ing function governing the fluctuations can guide poten- 
tial trade-offs and optimization in network synchroniza- 
tion. For systems with more general (non-linear) node 
dynamics, one can also expect that the synchronizabil- 
ity phase diagram will exhibit non-monotonic behavior 
as a function of the coupling strength and/or the delays 
[lOl . [ill . [20I [2l| . In real communication and information 
networks, the delays Tij are not uniform 4], but are af- 
fected by the network neighborhood and spatial distance. 
We currently investigate the impact of heterogeneous de- 
lays on network synchronization. 
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